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Objective  of  the  Course 

Provide  an  update  on  the  sensors,  methods,  and 
status  of  the  classification  of  military  munitions 
using  geophysical  methods 

•  Advanced  processing  of  data  collected  with  existing 
commercial  instruments 

•  Significant  improvements  possible  using  advanced  EMI 
sensors 
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•  There  are  over  3,000  sites  suspected  of  contamination 
with  military  munitions 

•  They  comprise  10s  of  millions  of  acres 

•  The  current  annual  cleanup  effort  is  on  the  order  of  1  % 
of  the  projected  total  cost 

•  To  make  real  progress  on  this  problem,  we  need  a  better 
approach 
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Defense  Science  Board  Task  Force  on  UXO 
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•  Classification  offers  the  chance  to  divide  anomalies  into 
those  caused  by  targets-of-interest  and  those  caused  by 
other  things 

•  Recognize  that  current  field  methods  involve  implicit 
discrimination 

♦  Mag  &  Flag  -  instrument  sensitivity  setting  and  human 
interpretation 

♦  Digital  Geophysics  -  threshold  selection;  what  is  a  target? 

•  Our  goal  is  a  principled,  data-based  approach  to  classify 
targets  as  either  “non-hazardous”  or  “targets  of  interest” 
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Standard  Processing  Stream 

The  standard  processing  stream  for  detection  and 
classification  of  munitions  using  geophysical  data 


1.  Data  Collection 


2.  Parameter  Estimation 
(Target  Attributes) 
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Adopted  from:  Pattern  Classification  (2nd  ed)  by  R.  O.  Duda,  P.  E.  Hart 
and  D.  G,  Stork,  John  Wiley  &  Sons,  2000 
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Extract  Parameters 


Adopted  from:  Pattern  Classification  (2nd  ed)  by  R.  O.  Duda,  P.  E.  Hart 
and  D.  G,  Stork,  John  Wiley  &  Sons,  2000 
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Are  These  Parameters  Useful  for  Classification? 
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'Are  These  Parameters  are  Useful  for 
Classification? 
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Electromagnetics  (EM): 
Fundamentals  and  Parameter  Extraction 

Stephen  Billings 


•  EMI  Fundamentals 

♦  How  EMI  sensors  work  and  what  they  measure 

♦  Principal  axis  polarizabilities 


•  Data  collection 

♦  Survey  and  mapping 

♦  Target  illumination 

•  Parameter  extraction 

♦  Dipole  inversion  to  determine  principal  axis  polarizabilities 
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Electromagnetic  Induction 


Advances  in  Classification  -  EMI  Fundamentals 


3 


Basic  EM  Concepts 


Primary  Field  from 
Transmit  Loop  excites 
Eddy  Currents  in  Object 


Receive  Loop  measures 
Induced  Field  due  to 
Eddy  Currents 


1.  The  primary  field  magnetizes  the 
buried  object  (similar  to  magnetics) 

2.  Abrupt  change  in  the  primary  field 
excites  eddy  currents  in  the  object. 

3.  Eddy  currents  diffuse  throughout  the 
object  and  decay  (basic  EM  response 
which  applies  to  all  metal  objects) 


transmit  current 
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Example  -  EMI  response  of  2”  diameter  carbon  steel  ball 
measured  with  TEMTADS  sensor,  compared  with  theory 


Time  (msec) 
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Stages  in  the  EMI  Response 
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i.  Target  magnetized  by 
primary  field 
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Stages  in  the  EMI  Response 
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1.  Target  magnetized  by 
primary  field 

2.  Primary  field  shuts  off 

•  Field  inside  target  cannot 
react  immediately 

•  Currents  form  on  surface 

•  Secondary  field  created 
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Stages  in  the  EMI  Response 


Decaying  induced  field 
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Stages  in  the  EMI  Response 
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1.  Target  magnetized  by 
primary  field 

2.  Primary  field  shuts  off 

•  Field  inside  target  cannot 
react  immediately 

•  Currents  form  on  surface 

•  Secondary  field  created 

3.  Induced  field  decays 
away  as  currents  diffuse 
into  target 
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Eddy  Current  Decay  Regimes 


•  Shape  of  EMI  response  determined  by  target  properties 

♦  Size,  shape,  thickness,  composition 

♦  Decay  times  from  1 0’s  of  psec  to  1 0’s  of  msec  can  provide 
information  useful  for  classification 
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Polarizability 

Standard  theory  represents  EMI  response  in  terms  of  an 
oscillating  magnetic  dipole  moment  that  is  induced  in  the 
target  by  an  oscillating  primary  field  H0exp(icot ) 

►  For  a  sphere,  this  dipole  moment 
is  oriented  in  the  same  direction 
as  the  primary  field,  with  strength 
proportional  to  the  primary  field 
amplitude  H0 

►  Proportionality  factor  is  the 
magnetic  polarizability  (3(a)) 

►  For  transient  EM,  polarizability  is 
Fourier/Laplace  transform  (3(t ) 
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Dipole  Response  Model 


•  For  arbitrary  target  shapes,  induced  dipole  moment  is 
typically  not  aligned  with  primary  field 

♦  Polarizability  is  second  rank  tensor  B (t)  relating  responses  in  x, 
y,  z  directions  to  primary  field  components  in  x,y,  z  directions 


/  ftityxx 

fi(p)xz 

B(f)  =  j 

(Oyy 

my* 

\mzX 

fiiQzz 

relates  dipole  moment 
induced  in  /  direction  to  primary 
field  component  in  j  direction 
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•  Suitable  yaw/pitch/roll  rotation  aligns  field  components 
with  target’s  three  orthogonal  principal  axes 

♦  B (t)  is  then  diagonal  &  elements  are  principal  axis  polarizabilities 
/?i>  fi2,  p3  corresponding  to  excitations  in  the  three  principal  axis 
directions 
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Principal  Axis  Polarizabilities 


•  Normalized  response  (polarizability)  for  excitation  in 
object’s  principal  axis  directions  are  the  fundamental  EMI 


attributes 

♦  UXO  items  are  symmetric, 
so  two  of  the  principal  axis 
responses  are  the  same. 
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Principal  Axis  Polarizabilities 


♦  UXO  items  are  symmetric, 
so  two  of  the  principal  axis 
responses  are  the  same. 

♦  Irregular  clutter  items  have 
three  different  principal  axis 
responses. 
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•  Normalized  response  (polarizability)  for  excitation  in 
object’s  principal  axis  directions  are  the  fundamental  EMI 
attributes  of  the  object 


ESTCP 


Examples  of  Polarizabilities 
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Stages  in  the  Classification  Process 


2.  Invert  data  using  dipole 
model  and  extract 
target  features 


3.  Classify  target  using 
decision  rules  applied 
to  target  features 
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3.  Classify  target  using 
decision  rules  applied 
to  target  features 


Advances  in  Classification  -  EMI  Fundamentals 


18 


December  1 , 2010 


Advances  in  Classification  Methods  for  Military  Munitions  Response 


•  Requires  a  geophysical  sensor  system  (based  on  either 
magnetometry  or  electromagnetic  induction  ) 

•  A  positioning  device  (e.g.  Global  Positioning  System, 
GPS) 

•  A  computer  for  digital  data  acquisition 


Digital  data 
acquisition 


Magnetometer 


Location  device 
(e.g.  GPS) 


Geophysical 
sensor 


Electromagnetic  sensor 
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•  Industry  standard  for  geophysical 
surveys  at  munitions  response 
sites 

•  Best  UXO  detection  performance 
at  1994-96  Jefferson  Proving 
Ground  demonstrations 

•  Measures  eddy  current  decay 
signal  over  four  time  gates 
centered  at  216,  366,  660,  and 
1266  psec 
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“Data”  profiles  at 
multiple  time-gates 


€BTCP 
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Mapped  Survey  Data  &  EM  Anomalies 


target  anomaly  92 
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Stages  in  the  Classification  Process 


Classify  target  using 
decision  rules  applied 
to  target  features 
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Calculating  Polarizabilities  from  EM  Data 


•  Data  collected  at  different  locations  over  a  target  sample 
different  combinations  of  the  principal  axis  responses 


•  We  can  use  the  dipole  response  model  to  sort  out  the 
different  contributions 


V(t)  -  |aonRnT^O^'R*^'T®(t) 
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Sampling  the  Full  EM  Response 


Measured  EMI  Response 
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•  EM  measurements  from  enough  locations  to  sample  all 
principal  axis  responses  is  inverted  using  the  dipole 
response  model  to  determine  the  polarizabilities 
♦  Inversion  also  determines  target  location,  depth  and  orientation 
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(Ps)  at  the  4  EM61  time 
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EM61  Target  Attributes 


»ESTCP 


1234  Based  on  polarizabilities 

(ps)  at  the  4  EM61  time 
gates 

Shape  attributes 

Symmetry  =  (P2-P3VP1 
Aspect  Ratio  =  !4(p2+p3)/p., 
(usually  gate  1) 


001  0.10  1.0  10  100 
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EM61  Target  Attributes 
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Time  (msec) 


Based  on  polarizabilities 
(Ps)  at  the  4  EM61  time 
gates 


Decay  attribute 
Decay  =  p1(4)/p1(1) 
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Forward  modeling  demonstration 
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Polarization 
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Grid  at  0.18  ms 

(time-channel  1) 


Grid  at  10  ms 
(time-channel  22) 


Time-decays 

(at  locations  marked  by 
black  and  magenta 
circles) 


Layout  of  the  data  view 


Observation  locations 
(white  dots) 


Polarizabilities 
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60  mm  mortar 

60  mm  mortar 

Vertical 

Horizontal,  North  facing 
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60  mm  mortar 

Horizontal,  North  facing 
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60  mm  mortar 
Depth  of  50  cm  (0.5  m) 


60  mm  mortar 

Depth  of  20  cm  (0.2  m) 


fir  Flft  W«  I 
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Note  the  change  in  spatial  extent  and  amplitude  of  the  data 
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60  mm  mortar  105  mm  projectile 

Aspect  =  0.38  Aspect  =  0.61 


Lower  aspect  at  early  time  results  in  sharper  anomaly  peak 
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60  mm  mortar  Stokes  mortar 

Aspect  =  0.38  Aspect  =  0.21 


Higher  aspect  at  early  time  results  in  double  humped  anomaly 
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Effect  of  item  decay  behavior 


60  mm  mortar 
Time-decay  =  5.1  ms 


Thinner  walled  item 

Time-decay  =  2  ms 


Note  the  more  rapid  decrease  in  late  time-data  with  the  faster  decay  rate 
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Parameter  extraction  demonstration 


Data  constrains  parameters 
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Model 
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Polarization 
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Depth  and 
orientation 


Survey 
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Layout  of  the  data  view 


Observed  Modeled 

data  data 


Layout  very  similar  to 
forward  modeling  data-view 
with  the  addition  of  a 
column  for  observed  data 

Objective  is  to  change  the 
model  parameters  until  the 
observed  and  modeled  data 
agree  as  closely  as  possible 
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Original  model  parameters  Adjusted  model  parameters 
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Model  from  last  step  Adjusted  model  parameters 

Depth  =  0.2  m  Depth  =  0.5  m 
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Model  from  last  step  Adjusted  model  parameters 

Size  =  1  Size  =  6 
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«4ESTCP 

Adjust  aspect  (and  size  a  little) 


Model  from  last  step 

Aspect  =  0.25 


Adjusted  model  parameters 

Size  =  6 


Adjust  time-decay 

(and  we’re  done) 

Model  from  last  step  Final  model  parameters 

Decay  =  10  ms  Decay  =  5.0  ms 
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Parameter  extraction  from  noisy  data 


Model  of  sensor  noise  Effect  of  position  errors 

2  mV  (at  first  time-channel)  4  cm  position  error 
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Effect  of  data  density 


Original  inversion  model 


4  .  .  riJ  i=  ft'  3  ■  I 


| 


M 


Modified  inversion  model 
Smaller  and  more  symmetric 

T—  x 

H  ■  3  £  K  ■  a  □  ■  a 


«oT 
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•  EMI  sensor  data  holds  information  about  target’s  size, 
shape,  thickness  and  material  composition 

•  Polarizabilities  along  target’s  principal  axis  directions 
fully  characterize  EMI  response 

♦  Basis  for  classification 

•  Target’s  principal  axis  polarizabilities  are  determined  by 
mathematically  inverting  EMI  data  collected  over  target 

♦  Requires  excitation  of  target  and  observation  of  response  from 
many  directions 

♦  Ability  to  constrain  polarization  tensor  parameters  depends  on 
quality,  density  and  diversity  of  collected  data 
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Stages  in  the  Classification  Process 


j ft,  .  jylilblJL 

®Jj  A 

265 

f 

yj2 

7  -'-A  „ 

1.  Acquire  EMI  data 
over  target 
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Classification  with  EM61  Data 

Dr.  Dean  Keiswetter 


•  Classification  Process 

♦  Methods 

♦  Product 

♦  Assessment 

•  EM61  Datasets 

♦  Elements  of  a  database 

♦  Analysis  environment  &  flow 

♦  Noise  issues  &  problems 

♦  Data  requirement  summary 

♦  Example  #1 :  Aberdeen  Proving  Ground 

♦  Example  #1 :  San  Luis  Obispo 

•  Concluding  comments 
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Classification  Objective 


We  want... 

(a)  to  identify  those  anomalies  that  are  definitely  clutter 
(they  cannot  possibly  be  UXO)  at  the  site 

(b)  a  principled  process  that  results  in  a  decision 


This  entire  process  is  not  magic... it  must 
make  sense,  be  physically  inspired,  and 
be  documented 
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ESTCP 


How  do  we  classify? 

Visually,  we  use  physical  attributes  such  as  size  &  shape 


Because  we  cannot  see  buried  objects,  we  must  rely  on 
attributes  determined  from  geophysical  data 
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The  attributes  that  we  use  are  based  on  the  intrinsic 
polarizabilities  or  measured  decay 


ID2 


101  r  12  3  4 


Time  (ms) 
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001 


EM61  Target  Attributes 


12  3  4 


0.10  1.0  10 
Time  (msec) 


Based  on  polarizabilities 
((3s)  at  the  4  EM61  time 
gates 

Size  attribute 
Size  <x  3^Ep  (gate  1 ) 


100 
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1234  Based  on  polarizabilities 

(ps)  at  the  4  EM61  time 
gates 

Shape  attributes 

Symmetry  =  (P2-P3VP1 
Aspect  Ratio  =  14(p2+p3)/p., 
(usually  gate  1) 


001  0.10  10  10  100 

Time  (msec) 
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001 


EM61  Target  Attributes 


12  3  4 


0.10  1.0  10 
Time  (msec) 


Based  on  polarizabilities 
(Ps)  at  the  4  EM61  time 
gates 

Decay  attribute 
Decay  =  p1(4)/p1(1) 


100 
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Given  attributes  for  a  site  with  a 
single  munitions  item 

General  Process: 

1 )  Visualize  attributes 

2)  Obtain  labels  (e.g., 
ground  truth  information) 

3)  Establish  boundaries  - 
this  is  the  classification 
piece 

It  can  be  this  easy... if  the 
features  are  separable 


10.00 


0.00  0.05  0.10  0.15  0-20 

Attribute  1 


Remember  the  goal:  identify  anomalies  that  are  not  UXO 
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EM61  Sensor  &  Data  Collection 


The  work  horse  in  the  UXO 
EMI  surveying  industry... 

Designed  and  manufactured 
by  Geonics  Limited 

•Produced  since  1993 
•Sold  in  15  countries 
•Number  of  sensors  -500 
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iteJESTCP 

Maps  =  EMI  +  GPS  +  Orientation 


& 


1_lev(01 


«r*1Jev[1] 

em61  Jev{3] 
em61  Jev_mode|0] 


Spatial 
registration. . 


Sensor 

data... 


Orientation 

data... 
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Geo-registered  Data  Inputs 


,M*5afll/; 

:40: 17>S44 


1  SPTNL, GGK, 184017.00, 112404, 3549 . 0798472 6, N, 07848 . 20885678, ¥, 1, 07, 1 . 9, EHT78 . 333 , 

2  $PTNL, AVR, 18jl016 . 90, +2 62 . 5359, Yaw, -1.55 10, Tilt, ,,6.68 1,3, 1.9, 7*0B  11/24/04  13 

3  SPTNL, GGK, 184017 . 10, 112404,3549 . 07984701 

4  SPTNL, AVR, 184017 . 00, +2 62 . 5359, Yaw, -1 . 56' 

5  SPTNL, GGK, 184017.20, 112404, 3549 . 0798463' 


11/24/04  13:40:17.020 


t.t  i.,  07, 1.9,  EHT78 .335,  M*5D  11/24/04  13:40:17.110 

GPS  raw  data  7*0F  11/24/04  13:40:17.133 

_  _  lqvv  ^ 01101  |,07,  1.9, EHT78. 334, H*58  11/24/04  13:40:17.214 

6  SPTWL, AVR, 184017. 10, +2 62. 5113, Yaw, -1.5074, Tilt,,, 6. 684, 3,1.9, 7*0C  11/24/04  13:40:17.237 


. . T. } POD SO . . .E 
y. ' .a. . .U.qRODSZ 
.  .  .  Ey.t  .ae.  .  .U.wR 
□□S> . . .Ey.*.g. . . 

U. tRODSw. . .Ey. ■ . 
g.  .u.-Rnnsi. . .e 
y. £ . a. ! . U. tPDDS® 
. . . Ey. » . a. ".V. zR 
□□S  . . .Ey.A.a. ! . 

V. -RCIOSJ. . . Ey . E . 
U.sFCDSr. . .E 


J4A. 


Tiltmeter  data 


Advances  in  Classification  -  Classification  with  EM61  Data 


1  s  52^ 11/24/04  13:40:17.048 

2  S  52>SHJ^24/04  13:40:17 
S  52t  11/24/04  13:40:17.248 

4  S  77"  _i_x_  I-348 

.448 
.548 

7  $  52  11/24/04  13:40:17.648 

8  S  52  11/24/04  13:40:17.748 

9  S  52  11/24/04  13:40:17.848 
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Georegistered  Database 
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Process  Overview 
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Measured  Data 
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Prioritized  Dig  List 

The  Classification  Product 

Rank  based  on  the  probability  that  the  anomaly  is  clutter 


■T 


Do  notdb 

Dig  Threshold 


DtK 


Target  ID# 


438 


23 

5 


694 


4 

305 


Rank  Category 


Overlaps 


Can  Analyze:  Likely  Clutter 


Can  Analyze:  Likely  Clutter 
Can  Analyze:  Likely  Clutter 


Can  Analyze:  Cannot  Decide 


Can  Analyze:  Cannot  Decide 
Can  Analyze:  Cannot  Decide 


Once  prioritized,  establishing  the  dig  threshold  is  critical 
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Performance  Assessment 


Receiver  Operating  Characteristic  Curve 
(only  exists  if  all  detected  targets  are  removed) 


o  poo 

Advances  in  Classification  -  Classification  with 


high  confidence 
clutter 


fion 


noo 


1000 


POOrj 


Ntimhflr  ot  L InnscftRsary  [Jigs 

yi61  Data 
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ESTCP 


Data  Analysis  Environment 


Oasis  montaj 
•High  performance 
database 
•Advanced  data 
processing 
•Dynamic  linking 
(maps,  data,  profiles, 
etc.) 

•  Professional  map 
production 
•Audit  trail 
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»ESTCP 


uxo 

60mm 

81mm 

105mm 
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»ESTCP 
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Batch  Model 


Sensor  type:  EM61MK2 

Database:  ®  Chronological  Q  Non  chronol 


Subset  database:  ®  Update 


O  Use  existing  Subset  database:  ©Update 


Site  Database  Target  Database  EM61 


♦  Name: 

*  Sensor  channel: 


Data 


♦  Sensor  height: 

O  Variable  channel: 
©Constant:  o.4i 

-  Orientation  Information  - 

Pitch  channel:  [ 

Roll  channel: 

♦  Heading  channel:  Heading 


Sensor  type:  EM61MK2 

Database:  ©  Chronological  0  Non  chronol 


O  Use  existing  Subset  database:  ©Update 


Site  Database  i  Target  Database  ]  EM61 


♦  Name: 

♦  Group: 

♦  ID  channel: 

♦  Start  ID: 

♦  End  ID: 

♦  Mask  channel: 


EM61  TARGETS,  gdb 


1 1307 
[Mask 


Sensor  type:  EM61MK2 

Database:  ©  Chronological  Q  Non  chronological 


O  Use  existing 


Site  Database  Target  Database  (EM6lj 


♦  Coil  size: 

♦  Coil  separation: 
Mode: 


|o.5xl  m 

d 

OD 

©4 

©  Gate  1 
O  Gate  2 
O  Gate  3 
O  Gate  4 
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Target  list  with  Attributes 


H  EM61  TARGETS. gdb 

niUB 

■amiKaaiigBiME 

EMBDEaBaaBHEnSlBg^Tl  ,au  14  nedian 

402771.7H  4369421.1 
402771 .81  4369421 .1 


402780.74  4369422.52 


10.  402784.534369422.89 


11  ■  402776.19  4369423.87 


402779.16  4369425.48 


13.  402787.064369425.53 


402774.61  4369425.94 


15.  402783.224369425.87 


402772.44  4369426.25 


17.  402780. 6E  4369427.1 


402789.32  4369427.71 


19.  402791.574369428.22 


402776. 8E  4369428.06 


21.  402784.524369428.! 


402773.84  4369428.1 


1402773.84  4369428.1 


J>0  L  1.14 

86  \28.55 


Tau_14_median 


*  Group: 

*  ID  channel:  |  FLAGJD 

*  Mask  channel:  Mask 


Subset  Database  - 

*  Name: 

*  Data  channel: 


APG IDF EM6 1 UXA EM6 1  • 


*  Gates  for  Tau  calc:  [l4 

Acceptance  threshold:  [o~ 

*  Amplitude  threshold:  0 
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127 

148 

375 
154 
124 
112 
267 
225 
378 
216 
196 
122 
108 
2  02 
196 
168 
157 


163 

166 


159 

248 


248  i 
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EM61  Attribute  Characteristics  at  APG 

Three  types  of  UXO:  60mm,  81  mm,  &  1 05mm 


60mm 

81mm 

105mm 

median 

mean 

std  dev 

median 

mean 

std  dev 

median 

mean 

std  dev 

Intrinsic  size 

2 

36 

265 

4 

9 

15 

8 

20 

30 

Model  error 

Decay  rate 

60mm 

81mm 

105mm 

median 

mean 

std  dev 

median 

mean 

std  dev 

median 

mean 

std  dev 

Intrinsic  size 

Model  error 

17 

21 

13 

14 

18 

10 

15 

18 

11 

Decay  rate 

60mm 

81mm 

105mm 

median 

mean 

std  dev 

median 

mean 

std  dev 

median 

mean 

std  dev 

Intrinsic  size 

Model  error 

1 1 

Decay  rate 

639 

643 

34 

633 

627 

35 

637 

633 

27 

These  intrinsic  polarizabilities  do  not  cluster... 
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For  each  Anomaly: 

Can  we  Classify  based  on  Polarizations? 


Measured 


Model 


02786  402787  402788 

Measured  Data 


GOOD  ENOUGH? 

◄ - 

Fit  Results 

Xftn):  4027S7.G5 

Y{m):  43$960$.w 

Depth  (m>:  9.05 

Theta  fine)  (’):  -15.9 
Phi  £Dbc)  276.4 
;Psi{R0t)n:  154.7 

1.623 
0  590 
0.92S 
0.961 
0-2 

105mm  H456  NEAT 


402787  402788 

Modeled  Data 


Forward  models  suggest  that  a  tremendous  amount  of 
information  is  contained  in  our  model  -  but  is  the  anomaly 
in  question  good  enough? 
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Model  Match  Fit  Quality 


Fit  quality  is  determined  by  the  mismatch  between  the 
data  and  dipole  model  fit  to  the  data 

♦  Reflects  ability  of  data  quality  to  support  inversion  and  estimation 
of  target  attributes  for  classification 


482; 

1 

1 

1 

r'? 

§ 

i 

'""A 

§ 

_ Zm 

£ 

; 

1 

177 

ft 

■4 

* 

079 

464  679 

456  679 

1 

458 

... 


I 


§  2  = 


Well  resolved  anomaly 
with  good  dipole  fit 
quality  (5%  fit  error) 

EM61  (gate  1)  data 
over  2.75”  rocket 
warhead  at  Badlands 
Bombing  Range 
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Parameter  Extraction  Issues 


679470 _ _ _ 679472 


679470 - * - 679472 

Measured  Data 


Line  spacing  does  not  provide  adequate 
sampling  of  the  intrinsic  polarizabilities 


679416 _ 679418 


X  27 

3 

X  259 

679416  679418 

Measured  Data 


Overlapping  signals 


§  3 
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(low  SNR) 
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Survey  Data  Quality 


Q) 

"l3 

-Q 


Good  survey 
quality  control  is 
important  for  ^  00 
classification 


<D 

E 

E 

> 

0) 


Same: 

site 

buried  objects 
sensor 

Different: 

signatures 

repeatability 

classification 
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Reliable  estimation  of  target  attributes  (polarizabilities) 
requires  very  high  quality  data  with  dipole  fit  error  less  than 
5-10% 

40 


g  30 

SNR  approaching  100  is  J 

required  for  classification,  f 20 

compared  to  ~5  for  detection  a 

10 

0 

1  10  100  1000  10000 
Signal  to  Noise  Ratio 
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•  Errors  in  recorded  sensor  locations  corresponding  to  EM 
data  can  substantially  increase  dipole  fit  error 

♦  It  is  very  hard  to  maintain  survey  geolocation  at  the  accuracy 
level  required  to  support  reliable  classification 
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Data  Requirement  Summary 


•  Inversion  requires  accurately  mapped  survey  data 
♦  Also  requires  sensor  orientation  and  vertical  position 

•  Data  density  and  spatial  extent  must  adequately  sample 
the  principal  axis  polarizabilities 

•  No  overlapping  signals 

•  SNR  needed  for  classification  >  SNR  needed  to  detect 
object 
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Attributes:  size  vs.  symmetry 
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Attributes:  UXO  vs  Clutter 

size  vs.  symmetry 


EM61  Cart 


RED  --  target  of  Interest 
Blue  --  munitions  debris 
Green  --  cultural  debris 


Advances  in  Classification  -  Classification  with  EM61  Data 


size  vs.  decay 
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Classification  Approaches... 


|^>  Intrinsic  polarizations 

H\  Intrinsic  polarizations  & 
v  Decay  rate 

[^Amplitude 


Decay  rate 
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EM61  Cart  Classification  Performance 


ROC  based  on 
Training  Labels 


EM61  Array 
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False  Negative 
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SLO  -  EM61  cart  no  slope  -  Anomaly  1444 
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ESTCP 


Camp  Sibert,  Alabama 


578,200  578,400  578,600  578,800  579,000 

UTM  Easting  (m) 

UTM  =  Universal  Transverse  Mercator 
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feCSTCP 
EM61  Mk2  Data  -  Man  Portable  Cart 
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Munitions  and  Clutter  Classes 


Munitions  Debris 


4.2-inch  Mortars 


Half  Shells 
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Data  Analysis  Objec 


Discriminate  4.2-inch  mortars  from  native  clutter 

♦  Characterization  (inversions) 

♦  Classification  (statistical  classifier  or  canonical  metrics) 


EM61  Mk2  Array  Data  Color  Scale  0-50  millivolts  10  meters  grid  lines 

EM61  Mk2  sensors  are  products  of  Geonics  Limited 
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Performance  based  on  Size  Estimates 
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:  -  □  1  High  Confidence  Non-TOI  (NO  DIG) 
□  2  Low  Confidence  NonTOI  (DIG) 

— i — m3  Can't  Analyze  (DIG) 

1000  □  4  Low  Confidence  TOI  (DIG) 

■  5  High  Confidence  TOI  (DIG) 
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^ESTCP 

Performance  Comparison 

Signal  Strength  vs.  Polarization-based  Size 

165  404 
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Respectable  classification  results  can  be  realized  using 
EM61  data  for  some  sites 

Target  size  and  decay  rate  were  the  best  attributes  for  the 
Camp  Sibert  and  Camp  San  Luis  Obispo  demonstrations 

Classification  performance  is  typically  poor  if  target  shape 
estimates  are  required  from  survey  mode  EM61  data 
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Advanced  EMI  Sensors 

Thomas  Bell 


•  Limitations  of  conventional  sensor  technology 

♦  Basic  design  considerations  for  advanced  sensors 

•  Advanced  sensor  systems 

•  Performance 

♦  Principal  axis  polarizabilities 

♦  Classification  performance 
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•  Classification  performance  using  conventional  EMI  sensor 
technology  is  limited  by  two  primary  factors 

♦  The  eddy  current  decay  cycle  is  not  fully  captured 

♦  Multi-cm  positioning  errors  inherent  to  field  survey  work 
compromise  the  accuracy  of  dipole  inversion  and  estimation  of 
target  attributes 

•  New  UXO-specific  technologies  which  avoid  these  problems 
are  being  developed  and  tested  under  SERDP  and  ESTCP 
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Conventional  Sensor  Technology 


Geonics  EM61  Mk2 

Industry  standard  for  geophysical 
surveys  in  munitions  response 
♦  Eddy  current  decay  signal  over  four 
time  gates  centered  at  216,  366,  660, 
and  1266  psec 

Best  UXO  detection  performance  at 
1994-96  Jefferson  Proving  Ground 
technology  demonstrations 

Efforts  to  process  survey  data  for 
shape-based  target  classification 
largely  unsuccessful 
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Target  Features  from  EM  Data 


Intrinsic  responses  (polarizabilities)  along  target’s  principal  axis 
directions  fully  characterize  EMI  signal 

♦  Requires  illumination  of  target  and  observation  of  response  from  all 
directions 
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Target  Features  from  EM  Data 


Intrinsic  responses  (polarizabilities)  along  target’s  principal  axis 
directions  fully  characterize  EMI  signal 

♦  Requires  illumination  of  target  and  observation  of  response  from  all 
directions 

♦  Cannot  be  accurately  determined  from  conventional  survey  quality 
EM  data  because  of  positioning  uncertainty 

♦  Advanced  sensors  use  fixed  coil  arrays  for  precise  positioning  of 
sensor  readings  to  allow  accurate  calculation  of  principal  axis 
polarizabilities 
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»ESTCP 


•  Significant  overlap  in  primary  and  secondary  polarizabilities 
from  EM61  survey  data  compared  to  advanced  sensors 
♦  Target  size  &  shape  attributes  washed  out 


Target  size,  symmetry  and  aspect  ratio  resolved  with  data 
from  advanced  sensors 


60mm  mortars  -  2009 
classification  demo  at 
Camp  San  Luis  Obispo 
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Polarizability 
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Target  Features  from  EM  Data 


•  Intrinsic  responses  (polarizabilities)  along  target’s  principal  axis 
directions  fully  characterize  EMI  signal 

♦  Determined  by  physical  properties  of  the  target 

♦  Eddy  current  decay  times  from  10’s  of  psec  to  10’s  of  msec 


Time  (msec) 
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Eddy  Current  Signal  Decay 


Intrinsic  responses  (polarizabilities)  along  target’s  principal  axis 
directions  fully  characterize  EMI  signal 

♦  Determined  by  physical  properties  of  the  target 

♦  Objects  with  roughly  comparable  size  and  shape  can  have  similar 
early  time  responses  but  different  late  time  responses 
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Advanced  Sensor  Configurations 


Multi-axis  sensors 
Target  directly  underneath  sees 
vertical  primary  field  with  vertical 
axis  coil  or  horizontal  field  with 
horizontal  axis  coil. 


Single  axis  arrays 
Target  directly  underneath  vertical 
axis  coil  sees  vertical  primary  field, 
oblique  fields  for  coils  off  to  sides. 
Includes  bi-static  transmit/receive 
combinations. 
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Survey  vs.  Cued  ID  Operation 


3ESTCP 


•  Classification  using  survey  data  from  moving  sensor  arrays  is 
challenging 

♦  Exciting  target  from  different  directions  and  observing  response  to 
late  times  takes  time 

♦  Data  density  along  survey  lines  suffers 

•  Most  current  systems  use  cued  identification  approach 

♦  Park  sensor  over  previously  flagged  anomaly,  collect  data  and 
move  on  to  next  anomaly 

♦  Several  hundred  targets  per  day  possible 
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Advanced  Sensor  Systems 


•  Multi-axis  sensor  systems 

♦  BUD  -  Berkeley  UXO  Discriminator  (Lawrence  Berkeley  Nat’l  Lab) 

♦  MetalMapper  (Geometries,  G&G  Sciences,  Snyder  Geoscience) 

♦  ALLTEM  (US  Geological  Survey) 

♦  EM63-3D  Mk2  (Geonics) 

•  Single  axis  arrays 

♦  TEMTADS  -  Transient  EM  Towed  Array  Discrimination  System  (US 
Naval  Research  Lab,  Nova  Research,  G&G  Sciences,  SAIC) 

•  Man-portable  and  handheld  derivatives 

♦  Handheld  BUD 

♦  Man-portable  vector  sensor  (G&G,  ERDC  Hanover,  Sky  Research) 

♦  Man-portable  and  handheld  TEMTADS 
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Berkeley  UXO  Discriminator  (BUD) 


•  3  orthogonal  transmit  coils  (1  m  square)  -  8  receive  coil  pairs 

♦  24  independent  Tx/Rx  measurements  of  transient  response  from 
140  to  1400  psec 

•  Survey  mode  (detection)  or  cued  ID 
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•  3-axis  transmit  coils  (30  cm  square)  -  10  receive  coil  pairs 
♦  Backpack  electronics 


Advances  in  Classification  -  Advanced  EMI  Sensors  14 


December  1, 2010 


7 


Advances  in  Classification  Methods  for  Military  Munitions  Response 


»ESTCP 


•  3  orthogonal  transmit  coils,  7  multiaxis  receive  cubes 
♦  69  Tx/Rx  measurements  of  transient  response 

•  Survey  mode  (detection)  or  cued  ID 

•  Commercially  available  (Geometries) 
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Man-Portable  Vector  (MPV)  Sensor 
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»ESTCP 


•  CW  traingular  waveform 
switched  between  three 
orthogonal  transmit  coils 

•  19  receiver  coil  configurations 

•  Survey  mode  operation  for 
detection  and  classification 


»ESTCP 


•  Three  orthogonal  transmit  coils  with  multi-axis  receive 

•  Survey  mode  detection/classification 

♦  Decay  to  25  msec  at  7.5  Hz  base  period,  7.3  msec  at  30  Hz 

•  Commercially  available  (Geonics) 
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•  Transient  EM  Towed  Array  Discrimination  System  (TEMTADS) 

•  2D  array  of  25  time  domain  EMI  sensors,  decay  times  from 
0.04  to  25  msec 

♦  325  unique  transmit/receive  combinations 

♦  Cued  ID 
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•  Man-portable  electronics  and  DAQ  package 

•  Cued  ID  using  grid  template  over  target 


♦  IMU  positioning  in  future 
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•  Full  5x5  array 


Cl 

E 


% 


CL 


0.01 


1.0 

Time  (msec) 


APG  Cal  Grid 
D1  clutter 


IQ1  r 

10°  r 

101  r 

10‘2  p 

10’3  r 


Advances  in  Classification  -  Advanced  EMI  Sensors 


22 


December  1, 2010 


11 


Advances  in  Classification  Methods  for  Military  Munitions  Response 


•  Full  5x5  array 

•  2x2  man-portable 


Time  (msec) 
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Principal  Axis  Polarizabilities 


Full  5x5  array 
2x2  man-portable 
Handheld  sensor 
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Polarizabilities 


APG  Cal  Grid 
K6  105  mm 
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Classification 
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D1  clutter 
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Metal  Mapper  cued  ID  vs.  EM61  survey 

♦  Camp  San  Luis  Obispo  Classification  Demonstration 

♦  Common  targets  (204  munitions,  776  clutter  items) 


Number  of  Excavated  Clutter  Hems 
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•  Classification  performance  using  conventional  EMI  sensor 
technology  is  limited  by  two  primary  factors 

♦  The  eddy  current  decay  cycle  is  not  fully  captured 

♦  Multi-cm  positioning  errors  inherent  to  field  survey  work 
compromise  the  accuracy  of  dipole  inversion  and  estimation  of 
target  attributes 

•  New  UXO-specific  technologies  which  avoid  these  problems 
are  being  developed  and  tested  under  SERDP  and  ESTCP 

♦  Results  from  technology  demonstrations  are  very  encouraging 

■  Aberdeen  &  Yuma  Proving  Ground  Standardized  Test  Sites 

■  Former  Camp  Sibert,  Camp  San  Luis  Obispo,  Camp  Butner  live  site 
classification  demonstrations 
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Classification  with  Advanced  EMI 
Sensor  Data 


Dr.  Dean  Keiswetter 


•  Review 

•  Data  analysis  environment  /  processing  overview 

•  Classification 

♦  Approach  and  features 

♦  Aberdeen  Proving  Ground 

♦  San  Luis  Obispo 

•  Multiple  source  situations 

•  Closing  comments 
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Review:  Factors  Affecting  Classification 
Performance  with  EM 61  data 

•  Limited  capability  for  target  classification  in 
survey  mode 

♦  Analog  smoothing  distorts  signal  shape 

♦  Limited  decay  time  coverage 

♦  Centimeter-level  sensor  positioning  uncertainty 
degrades  target  parameter  estimates 

•  Towed  arrays  have  limited  target  illumination  with 
transmitters  operated  simultaneously 
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Review:  EMI  Sensors  designed  for 


Classification 


In  order  to  observe  the  complete  EM  response  pattern  the 
object  must  be  excited  and  measured  from  all  directions 

The  new  EMI  technologies  accomplish  this  with  multi-axis 
coil  sensors  or  single  axis  coil  arrays 


Multi-axis  coil  array  Single  axis  planar  array 
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ESTCP 


Data  Analysis  Environment 


Oasis  montai 
•High  performance 
database 
•Advanced  data 
processing 
•Dynamic  linking 
(maps,  data,  profiles, 
etc.) 

•  Professional  map 
production 
•Audit  trail 
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TEMTADS 


25  transmitters/25  receivers 

♦  Park  over  location  estimated  by  previous  survey 

♦  Fire  transmitter  1 ,  record  on  all  25  receivers 

♦  Fire  transmitter  2,  record  on  all  receivers 

♦  and  so  on  ... 
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•  3  orthogonal  transmitters  &  7  three-axis  receivers 

♦  Park  over  location  estimated  by  previous  survey 

♦  Fire  transmitter  1,  record  on  all  21  receivers 

♦  Fire  transmitter  2,  record  on  all  receivers 

♦  Fire  transmitter  3,  record  on  all  receivers 
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»ESTCP 
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ESTCP 


Aberdeen  Proving  Ground 
Classification  Example 
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Features  for  discrimination 
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Statistical  classification 
•Size 

•Time-decay 

•Shape 

•Asymmetry 

Library  based  classification 
Template  matching 
“Expert”  identification 
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feCSTCP 

Classification  using  a  rule-based  approach 


10 

Time  (msec) 


“Library  match”  metric, 
compares 

1.  Primary  polarizability  (PO 

2.  Ratio  secondary  to  primary  (p2/  p.,) 

3.  Ratio  tertiary  to  primary  (p3/ p.,) 
for  targets  of  interest 
3-criteria  (p,;  p2;  p3) 

2-criteria  (p,;  p2) 

♦  Robust  to  p3  for  weak  signals 
Decision  boundary  chosen  to 
accommodate  training  data 
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Axial  Symmetry 


•  Targets  with  axially  symmetric  response  that  do  not 
match  known  munitions  included  in  “can’t  decide” 

♦  Hedge  against  unexpected  munitions  (e.g.  3”  Stokes  mortar) 


Target  428 


Target  140 


0.01  0.1  1.0  10  100 


0.01  0.1  1.0  10  100 
Time  (msec) 


Time  (msec) 
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Can’t  Analyze  &  Can’t  Decide 

CanXAnalyze 

Lack  of  data  due  to  a  sensor-specific  data  gap 
Inversion  fails  to  converge 

Inversion  produces  unphysical  parameters  (depths  >2m  or 
negative  polarizabilities) 

Can’t  Decide 

Low  SNR 

Multiple  sources  suspected  (overlapping  signatures) 
Axially  symmetric*  (but  does  not  match  library) 

Buffer  zone* 

*TEMTADS  &  MetalMapper 
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Stable  Polarizations: 
the  Key  to  classification 


Blue  ->  Unknown 


Grey  ->  Library  from  APG 


60mm 


100 


81mm 


Time  (ms) 
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Stable  Polarizations: 
the  Key  to  classification 


Blue  ->  Unknown 


Grey  ->  Library  from  AF  | 


60mm 


367 


81mm 


0.1  1.0  10.0 


i  ?  1  I 
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Targets  of  Interest  (TOI) 


Camp  SLO:  T argets  of  Interest 


Blue  ->  seed 


.ill 


Green  ->  native 


5  c/t*  f  4/^  Am/ 

1 5  XQ 


2.36-in  4.2-in  60mm  81mm  3-in  37mm  5-in 

Munitions  Type 


n  Rocket  Warhead 
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ESTCP 


non  Targets  of  Interest... 
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Camp  SLO:  non-T argets  of  Interest 


1800 
1600 
1400 
1200 
iiooo 
I  800 
Z  600 
400 
200 
0 

Construction  Debris 


Munitions  Debris  No  Contact 

Clutter  Category 


1550 

] 

- 

— 

^  3 

»ESTCP 
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Calibration  Line  &  Pit  (on  site): 

60mm,  81mm,  2.36-in  rocket,  and 
4.2-in  rocket 

Survey  data:  233  anomalies 
excavated  for  training  purposes 

Classification  labels 


200 


42 
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TEMTADS  Polarizations 


»ESTCP 


100.0000 

10.0000 

1 .0000 

■g  0.1000 


0.0100  r 

0.0010  r 

0.0001  [ _ , _ 

0.01 

Advances  in 


0.10  1.00  10.00 
Time  (ms) 

Classification  -  Classification  with 


Advanced  Sensors 
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Both  EMI  Systems 
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feCSTCP 

Exploring  the  Value  of  Training  Labels 

•No  on-site  training 
•Assume  knowledge  of 
TOI  munitions  type 
•Utilize  prior  polarization 
library 


#uxo 

#  Digs 

50 

50 

100 

100 

183 

200 

216 

250 

221 

300 

11 

The  Rest 
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n*ESTCP 

Anomaly  Misclassified  Using  EM61 
Data  Was  Correctly  Typed 


MASTER  ID 

DECISION 

STATISTIC 

Rank 

Category 

Overlaps 

Type 

Easting  (m) 

Northing  (m) 

Depth (m) 

Azimuth 

1444 

413 

0.0233 

0.0231 

1152 

1153 

0 

0 

^ - ^ 

C  60mm  J) 

^  236m 

705441.6 

705275.41 

3913801.47 

3913921.9 

0.282 

0.326 

46.33 

137.07 

448 

60 

0.0227 

0.0226 

1154 

1155 

3 

3 

0 

0 

2.36in 

60mm 

705281.8 

705157.46 

3913666.61 

3913820.49 

0.28 

0.302 

7.53 

222.83 
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Implementation  Challenges 


Challenges  include: 

a.  multiple  objects  within  the  sensors  view 

b.  incorrect  cued  locations 

c.  poor  or  incomplete  ground  truth  information 
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Non-Inversion  Approaches 


Non-traditional  approaches  are  being  pursued  by  a  number  of 
research  firms. 


Objective(s)  include: 

•  Detect  and  classify  using  survey  data 

•  Increase  speed  of  calculations 

•  Decrease  noise  sensitivity 

•  Require  fewer  measurement  systems  (such  as  geolocation) 

•  Reduce  data  demands 

Key:  Intelligent  use  of  the  output 
from  multi-axis  loop  sensors 
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Multiple  Source  Situations 


The  munitions  debris  on 
the  left  was  5cm  below 
the  surface  and  21cm 
above  a  60mm  projectile 

The  measured  anomaly 
on  the  right,  however, 
provides  little  to  no 
evidence  that  multiple 
targets  are  present 


EMI  Data 


1  2 

Distance  (m) 
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Multiple  Source  Situations 

Complex  anomalies  are  not  uncommon... 

Electromagnetic  Data  from  Aberdeen  Proving  Ground 


mV 

b307_fin 


50  meters 
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Because  of  these  the  EMI  sensor  advancements 

>  multiple  Tx  and  Rx  pairs, 

>  rigid  construction  and 

>  broadband  digital  electronics 

we  can  now  attack  multi-source  scenarios... 
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»ESTCP 


Multi-source  Solver  (MM-1662) 


1. 

2. 

3. 

4. 


Determine  number  and 
location  of  sources 
contributing  to  measured 
data 

Form  clusters 

Perform  multi-source 
simultaneous  inversion 

Classify  based  on  inverted 
intrinsic  polarizabilities 
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Solver  Documentation  (*.pdf) 


»ESTCP 


Elevation  View 
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>ESTCP 


•  Realizing  unprecedented  classification  performance 

♦  Multiple  firms  and  multiple  sensors 

♦  Hardware  design  improvements 

■  Multi-coil  arrays  in  fixed  geometries 

■  Broad  bandwidth 

♦  Analysis  software  improvements 

■  Multi-source  solvers 


Metal  Mapper  systems  are  available  for  purchase  by  Geometries  or 
GFE  through  the  Corps  of  Engineers 
Analysis  software  is  available  through  Geosoft  and  Geometries 
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A  tidal  wave  of 
technology  is  here... 

sensors  and  software 
lead  the  way, 

but  operations  and 
mission  clarity  are 
close  behind. 
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Concluding  Thoughts 

Herb  Nelson 


»ESTCP 


Objective  of  the  Course 


Provide  an  update  on  the  sensors,  methods,  and 
status  of  the  classification  of  military  munitions 
using  geophysical  methods 

•  Advanced  processing  of  data  collected  with  existing 
commercial  instruments 

•  Significant  improvements  possible  using  advanced  EMI 
sensors 


Advances  in  Classification  -  Concluding  Thoughts 


2 


December  1 , 2010 


1 


Advances  in  Classification  Methods  for  Military  Munitions  Response 


Breakdown  of  $200M  FUDS  MMRP 


CO 

0 
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0 

cr 
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Breakdown  of  $200M  FUDS  MMRP 


Disposal 


CO 

0 
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Breakdown  of  $200M  FUDS  MMRP 


Assessment  Mapping  Removal  Removal  Removal  &  Remediated 

Disposal 


co 

0 

>- 


ro 

0 

cr 
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Implementation  Approaches 


•  Hazard-based  dig  decision 

♦  High  confidence  non-hazardous  anomalies  remain  in  the  ground 

♦  Remaining  anomalies  are  dug 


•  Hazard-based  dig  protocol 

♦  High  confidence  non-hazardous  anomalies  dug  with  one  UXO 
tech  supervising  a  team  of  lower-cost  diggers 

♦  Remaining  anomalies  are  dug  with  usual  procedures  (UXO 
personnel  and  safety  equipment) 

Approach  would  be  site  dependant  and  determined 
by  the  site  team 
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•  Requires  transparent  process  involving  explicit, 
documented  classification 

•  Continued  collaboration  with  stakeholders-  Advisory 
Group  and  beyond 

•  Need  to  start  thinking  about  things  like  QC  methods 
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Meeting  DoD's  Environmental  Challenges 


•  SERDP  &  ESTCP  Information  at  One  Location 

♦  Easy  access  to  all  information 

■  Funding  opportunities 

■  Investigator  resources 

■  Research  results 

♦  Highlights  program  areas  and  initiatives 

♦  Platform  for  technology  transfer:  Tools  and  Training 

serdp-estcp.org 
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